Method and apparatus for monitoring dynamic cardiovascular function using n-dimensional representatives of critical functions

ABSTRACT

A method, system, apparatus and device for the monitoring, diagnosis and evaluation of the state of a dynamic pulmonary system is disclosed. This method and system provides the processing means for receiving sensed and/or simulated data, converting such data into a displayable object format and displaying such objects in a manner such that the interrelationships between the respective variables can be correlated and identified by a user. This invention provides for the rapid cognitive grasp of the overall state of a pulmonary critical function with respect to a dynamic system.

CROSS REFERENCE TO RELATED APPLICATIONS

This patent application is a continuation-in-part patent application ofcopending U.S. patent application Ser. No. 09/457,068, which was filedon Dec. 7, 1999 and of copending Provisional Patent Application Ser. No.60/328,880, filed on Oct. 12, 2001. Priority is hereby claimed to allcommon material disclosed in these pending patent cases.

FEDERAL RESEARCH STATEMENT

Some of the technology described in this patent application was fundedin part by National Institute of Health Grant No. 1R24 HL 64590 and NASAGrant No. NGTA40101.

BACKGROUND OF INVENTION

1. Field of the Invention

This invention relates to the visualization, perception, representationand computation of data relating to the attributes or conditionsconstituting the health state of a dynamic system. More specifically,this invention relates to the display and computation of pulmonary andcardiovascular data, in which variables constituting attributes andconditions of a dynamic physiological system can be interrelated andvisually correlated in time as three-dimensional objects.

2. Description of the Related Art

A variety of methods and systems for the visualization of data have beenproposed. Traditionally, these methods and systems fail to present in areal-time multi-dimensional format that is directed to facilitating auser's analysis of multiple variables and the relationships between suchmultiple variables. Moreover, such prior methods and systems tend not tobe specifically directed to display of a patient's cardiovascular andpulmonary system by showing such variables as blood pressure, bloodflow, vascular tone and the like. Prior methods typically do not processand display data in real-time, rather they use databases or spatialorganizations of historical data. Generally, they also simply plotexisting information in two or three dimensions, but without usingthree-dimensional geometric objects to show the interrelations betweendata. Often previous systems and methods are limited to pie charts,lines or bars to represent the data. Also, many previous systems arelimited to particular applications or types of data. The flexibility andadaptability of the user interface and control is typically verylimited, and may not provide flexible coordinate systems andhistorical-trend monitors. Other systems, which have a flexible userinterface, generally require substantial user expertise in order tocollect and evaluate the data, including the pre-identification of dataranges and resolution. Another common limitation of previous systems andmethods is that they provide only a single or predetermined viewpointfrom which to observe the data. Typically, prior systems and methods donot provide data normalcy frameworks to aid in the interpretation of thedata. Furthermore, most prior methods use “icons,” shapes, lines, bars,or graphs.

The state-of-the-art for pulmonary monitored data in both traditionalintensive care units and anesthesiology units is the wave form andnumeric display monitors. These monitors represent information indiscrete variables associated with each sensor using the displayparadigm of a “single-sensor-single-indicator-display.” As a result,current monitors lack integrated data. The anesthesiologist is left tocollect and process all of the different data herself in a timelymanner.

The anesthesiologist is faced with a fire-hose of information duringevery case. Numerical data, waveforms, control dials, ventilatorbreaths, heart rate beeps, and alarm sounds bombard the anesthesiologistfrom all directions. During an uneventful case, the anesthesiologistsuccessfully monitors all the available data at a low cognitive level.However, should an unexpected event happen, the anesthesiologist mustquickly ascertain if a problem exists, the causality, and the neededcorrective action. Should the anesthesiologist fail at any of thesetasks, an error will occur. Human error is associated with more than 80%of critical anesthesia incidents and more than 50% of anesthetic deaths.

Studies have shown that integrated information represented in agraphical metaphor does reduce detection time and time to accuratelydiagnose a problem in anesthesia. However, the graphical metaphorsstudied to date tend to be complex and lack usability testing to drivedesigns. As a result, the graphical interface may inadvertently increasethe cognitive workload of the anesthesiologists.

For general background material, the reader is directed to U.S. Pat.Nos. 3,908,640, 4,193,393, 4,464,122, 4,519,395, 4,619,269, 4,752,893,4,772,882, 4,813,013, 4,814,755, 4,823,283, 4,832,038, 4,875,165,4,880,013, 4,915,757, 4,930,518, 4,989,611, 5,012,411, 5,021,976,5,103,828, 5,121,469, 5,162,991, 5,222,020, 5,224,481, 5,262,944,5,317,321, 5,425,372, 5,491,779, 5,568,811, 5,588,104, 5,592,195,5,596,694, 5,626,141, 5,634,461, 5,751,931, 5,768,552, 5,774,878,5,796,398, 5,812,134, 5,830,150, 5,836,884, 5,913,826, 5,923,330,5,961,467, 6,042,548, and 6,090,047 each of which is hereby incorporatedby reference in its entirety for the material disclosed therein.

As this disclosure employs a number of terms, which may be new to thereader, the reader is directed to the applicants' definitions section,which is provided at the beginning of the detailed description section.

SUMMARY OF INVENTION

It is desirable to provide a method, system, and apparatus, whichfacilitates the rapid and accurate analysis of complex and quicklychanging data. Moreover, it is desirable that such a system and methodinclude a graphic element that depicts the status of a patient'scardiovascular system by graphically showing blood pressure, blood flow,vascular tone and other cardiovascular variables. It is important thatsuch a graphic element provide an anesthesiologist with the means toquickly assess the patient's status. It is also desirable that theelement by comprised of subcomponents, which are linked together to showthereby the relationships of the various cardiovascular and pulmonaryvariables. Also, it is desirable that system and method be capable ofanalyzing time based, real-time, and historical data and that it be ableto graphically show the relationships between various data.

Research studies have indicated that the human mind is better able toanalyze and use complex data when it is presented in a graphic, realworld type representation, rather than when it is presented in textualor numeric formats. Research in thinking, imagination and learning hasshown that visualization plays an intuitive and essential role inassisting a user associate, correlate, manipulate and use information.The more complex the relationship between information, the morecritically important is the communication, including audio andvisualization of the data. Modern human factors theory suggests thateffective data representation requires the presentation of informationin a manner that is consistent with the perceptual, cognitive, andresponse-based mental representations of the user. For example, theapplication of perceptual grouping (using color, similarity,connectedness, motion, sound etc.) can facilitate the presentation ofinformation that should be grouped together. Conversely, a failure touse perceptual principles in the appropriate ways can lead to erroneousanalysis of information.

The manner in which information is presented also affects the speed andaccuracy of higher-level cognitive operations. For example, research onthe “symbolic distance effect” suggests that there is a relationshipbetween the nature of the cognitive decisions (for example, is the dataincreasing or decreasing in magnitude?) and the way the information ispresented (for example, do the critical indices become larger orsmaller, or does the sound volume or pitch rise or fall?). Additionally,“population stereotypes” suggest that there are ways to presentinformation that are compatible with well-learned interactions withother systems (for example, an upwards movement indicates an increasingvalue, while a downwards movement indicates a decreasing value).

Where there is compatibility between the information presented to theuser and the cognitive representations presented to the user,performance is often more rapid, accurate, and consistent. Therefore, itis desirable that information be presented to the user in a manner thatimproves the user's ability to process the information and minimizes anymental transformations that must be applied to the data.

Therefore, it is the general object of this invention to provide amethod and systems for presenting a three-dimensional visual and/orpossibly an audio display technique that assists a doctor in themonitoring of a patient's cardiovascular and pulmonary function.

It is a further object of this invention to provide a method and systemthat assists in the monitoring of a patient's cardiovascular and/orpulmonary system through the use of a three-dimensional graphic element.

It is another object of this invention to provide a method and systemthat assists in the management of anesthesia care of patients, bypresenting a display, which quickly shows the relationships of variouscardiovascular or pulmonary variables.

It is a still further object of this invention to provide a method andsystem that assists in the determination of the “health” of a dynamiccardiovascular or pulmonary system, by providing visual informationrelated to the nature or quality of the soundness, wholeness, orwell-being of the system as related to historical or normative values.

Another object of this invention is to provide a method and system thatassists in the determination of the functioning of a cardiovascular orpulmonary system by measuring the interaction among a set of“vital-signs” normally associated with the health of the cardiovascularsystem.

A still further object of this invention is to provide a method andsystem, which provides the gathering and use of sensor measured data, aswell as the formatting and normalization of the data in a formatsuitable to the processing methodology.

A further object of this invention is to provide a method and system,which organizes a cardiovascular or pulmonary system's data intorelevant data sets or critical functions as appropriate.

Another object of this invention is to provide a method and system,which provides a three-dimensional health-space for mapping thecardiovascular or pulmonary system data.

It is another object of this invention to provide a method and system,which provides three-dimensional objects that are symbols of thecritical functioning of the cardiovascular or pulmonary system beingmonitored.

It is an object of this invention to provide a method and system thatshows the relationships between several critical functions that a userwishes to monitor.

It is a further object of this invention to provide a method and systemthat permits an integrated and overall holistic understanding of thecardiovascular or pulmonary process being monitored.

A further object of this invention is to provide a method and systemwhere three-dimensional objects are built from three-dimensional objectprimitives, including: cubes, spheres, pyramids, n-polygon prisms,cylinders, slabs.

A still further object of this invention is to provide a method andsystem, wherein three-dimensional objects are placed within health-spacebased on the coordinates of their geometric centers, edges, vertices, orother definite geometric variables.

It is a further object of this invention to provide a method and system,which has three-dimensional objects that have three spatial dimensions,as well as geometric, aesthetic and aural attributes, to permit themapping of multiple data functions.

It is another object of this invention to provide a method and system,which shows increases and decreases in data values using changes inlocation, size, form, texture, opacity, color, sound and therelationships thereof in their context.

It is a still further object of this invention to provide a method andsystem, wherein the particular three-dimensional configuration ofthree-dimensional objects can be associated with a particular time andhealth state.

A still further object of this invention is to provide a method andsystem that permits the simultaneous display of the history of dataobjects.

Another object of this invention is to provide a method and system thatprovides for the selection of various user selectable viewports.

It is a further object of this invention to provide a method and systemthat provides both a global and a local three-dimensional coordinatespace.

It is another object of this invention to provide a method and systemthat permits the use of time as one of the coordinates.

It is a still further object of this invention to provide a method andsystem that provides a reference framework of normative values fordirect comparison with the measured data.

It is a further object of this invention to provide a method and systemwhere normative values are based on the average historical behavior of awide population of healthy systems similar to the system whose health isbeing monitored.

A further object of this invention is to provide a method and systemthat provides viewpoints that can be selected to be perspective views,immersive Virtual Reality views, or any orthographic views.

Another object of this invention is to provide a method and system thatpermits the display of a layout of multiple time-space viewpoints.

A still further object of this invention is to provide a method andsystem that provides for zooming in and out of a time and/or spacecoordinate.

It is another object of this invention to provide a method and systemthat permits temporal and three-dimensional modeling of data “health”states based on either pre-recorded data or real-time data, that is asthe data is obtained.

Another object of this invention is to provide a method and system thatpresents the data in familiar shapes, colors, and locations to enhancethe usability of the data.

A still further object of the invention is to provide a method andsystem that uses animation, and sound to enhance the usefulness of thedata to the user.

It is an object of this invention to provide a method and system for themeasurement, computation, display and user interaction, of complex datasets that can be communicated and processed at various locationsphysically remote from each other, over a communication network, asnecessary for the efficient utilization of the data and which can bedynamically changed or relocated as necessary.

It is a still further object of this invention to provide a method andsystem for the display of data that provides both a standard and acustomized interface mode, thereby providing user and applicationflexibility.

These and other objects of this invention are achieved by the method andsystem herein described and are readily apparent to those of ordinaryskill in the art upon careful review of the following drawings, detaileddescription and claims.

BRIEF DESCRIPTION OF DRAWINGS

In order to show the manner that the above recited and other advantagesand objects of the invention are obtained, a more particular descriptionof the preferred embodiment of the invention, which is illustrated inthe appended drawings, is described as follows. The reader shouldunderstand that the drawings depict only a preferred embodiment of theinvention, and are not to be considered as limiting in scope. A briefdescription of the drawings is as follows:

FIG. 1 a is a top-level representative diagram showing the dataprocessing paths of the preferred embodiment of this invention.

FIG. 1 b is a top-level block diagram of the data processing flow of thepreferred embodiment of this invention.

FIG. 1 c is a top-level block diagram of one preferred processing pathof this invention.

FIG. 1 d is a top-level block diagram of a second preferred processingpath of this invention.

FIGS. 2 a, 2 b, 2 c, and 2 d are representative 3-D objects representingcritical functions.

FIG. 3 is a representation of data objects in H-space.

FIGS. 4 a and 4 b are representative views of changes in data objects intime.

FIGS. 5 a, 5 b, 5 c, 5 d, 5 e, 5 f, 5 g and 5 h are representative viewsof properties of data objects provided in the preferred embodiment ofthis invention.

FIG. 6 shows a 3-D configuration of the objects in H-space in thepreferred embodiment of the invention.

FIG. 7 shows H-space with a time coordinate along with local-spacecoordinates.

FIGS. 8 a and 8 b show the global level coordinate system of thepreferred embodiment of this invention.

FIGS. 9 a and 9 b show various viewpoints of the data within H-space inthe preferred embodiment of this invention.

FIG. 10 shows the transformation of an object in space in context, witha reference framework, in the preferred embodiment of this invention.

FIG. 11 a shows the zooming out function in the invention.

FIG. 11 b shows the zooming in function in the invention.

FIGS. 12 a and 12 b show a 3-D referential framework of normativevalues.

FIG. 13 shows the interface modes of the preferred embodiment of thisinvention.

FIG. 14 is a hardware system flow diagram showing various hardwarecomponents of the preferred embodiments of the invention.

FIG. 15 is a software flow chart showing the logic steps of a preferredembodiment of the invention.

FIG. 16 is a software block diagram showing the logic steps of the imagecomputation and rendering process of a preferred embodiment of theinvention.

FIG. 17 is a photograph of the 3-dimensional display of a preferredembodiment of the invention.

FIG. 18 is a close-up front view of the cardiac object and theassociated reference grid of a preferred embodiment of the invention.

FIG. 19 is a view of the front view portion of the display of apreferred embodiment of the present invention showing the cardiac objectin the foreground and the respiratory object in the background.

FIG. 20 is a view of the top view portion of the display of a preferredembodiment of the present invention showing the cardiac object towardthe bottom of the view and the respiratory object toward the top of theview.

FIG. 21 is a view of the side view portion of the display of a preferredembodiment of the present invention showing the cardiac object to theleft and the respiratory object to the right.

FIG. 22 is a view of the 3-D perspective view portion of the display ofa preferred embodiment of the invention showing the cardiac object inthe left foreground and the respiratory object in the right background.

FIG. 23 a is a view of the preferred graphic element of this inventionin a normal cardiovascular system.

FIG. 23 b is a view of the preferred graphic element of this inventionin a cardiovascular system showing anaphylaxis.

FIG. 23 c is a view of the preferred graphic element of this inventionin a cardiovascular system showing hypovolemia.

FIG. 23 d is a view of the preferred graphic element of this inventionin a cardiovascular system showing bradycardia.

FIG. 23 e is a view of the preferred graphic element of this inventionin a cardiovascular system showing ischemia.

FIG. 23 f is a view of the preferred graphic element of this inventionin a cardiovascular system showing pulmonary embolism.

FIG. 24 is a view of the preferred reference grid of this embodiment ofthe invention.

FIG. 25 is a view of the preferred reference grid showing objectplacement in this preferred embodiment of the invention.

FIG. 26 is a view of the preferred reference grid showing the functionalobject relationships in this preferred embodiment of the invention.

FIG. 27 is a representative three-dimensional object used in the presentpreferred embodiment of the invention.

FIG. 28 is a representative view of the normalization of the presentpreferred embodiment of the invention.

FIG. 29 is an integrated view showing numeric information in the presentpreferred embodiment of the invention.

FIG. 30 is a view showing the addition of slopes to show the restrictionof blood vessels.

FIG. 31 shows a an embodiment of a pulmonary metaphor 3100 withanatomical representations.

FIG. 32 shows an embodiment of a pulmonary metaphor where variousfeatures are abnormal.

FIG. 33 shows an embodiment of a pulmonary metaphor where the intrinsicPEEP has increased.

FIG. 34 shows a preferred embodiment of a pulmonary metaphor.

FIG. 35 shows an embodiment of a pulmonary metaphor wherein the patienthas pulmonary fibrosis.

FIG. 36 shows an alternative embodiment of a pulmonary metaphor whereinthe patient has pulmonary fibrosis.

FIG. 37 shows an embodiment of a pulmonary metaphor wherein the patienthas emphysema.

FIG. 38 shows an alternative embodiment of a pulmonary metaphor whereinthe patient has emphysema.

DETAILED DESCRIPTION

This invention is a method, system and apparatus for the visual displayof complex sets of dynamic data. In particular, this invention providesthe means for efficiently analyzing, comparing and contrasting data,originating from either natural or artificial systems. In its mostcommon use the preferred embodiment of this invention is used to producean improved cardiovascular or pulmonary display of a human or animalpatient. This invention provides n-dimensional visual representations ofdata through innovative use of orthogonal views, form, space,frameworks, color, shading, texture, transparency, sound and visualpositioning of the data. The preferred system of this invention includesone or a plurality of networked computer processing and display systems,which provide real-time as well as historical data, and which processesand formats the data into an audio-visual format with a visualcombination of objects and models with which the user can interact toenhance the usefulness of the processed data. While this invention isapplicable to a wide variety of data analysis applications, oneimportant application is the analysis of health data. For this reason,the example of a medical application for this invention is usedthroughout this description. The use of this example is not intended tolimit the scope of this invention to medical data analysis applicationsonly, rather it is provided to give a context to the wide range ofpotential application for this invention.

This invention requires its own lexicon. For the purposes of this patentdescription and claims, the inventors intend that the following terms beunderstood to have the following definitions.

An “artificial system” is an entity, process, combination of humandesigned parts, and/or environment that is created, designed orconstructed by human intention. Examples of artificial systems includemanmade real or virtual processes, computer systems, electrical powersystems, utility and construction systems, chemical processes anddesigned combinations, economic processes (including, financialtransactions), agricultural processes, machines, and human designedorganic entities.

A “natural system” is a functioning entity whose origin, processes andstructures were not manmade or artificially created. Examples of naturalsystems are living organisms, ecological systems and various Earthenvironments.

The “health” of a system is the state of being of the system as definedby its freedom from disease, ailment, failure or inefficiency. Adiseased or ill state is a detrimental departure from normal functionalconditions, as defined by the nature or specifications of the particularsystem (using historical and normative statistical values). The healthof a functioning system refers to the soundness, wholeness, efficiencyor well being of the entity. Moreover, the health of a system isdetermined by its functioning.

“Functions” are behaviors or operations that an entity performs.Functional fitness is measures by the interaction among a set of“vital-signs” normally taken or measured using methods well known in theart, from a system to establish the system's health state, typically atregular or defined time intervals.

“Health-space” or “H-space” is the data representation environment thatis used to map the data in three or more dimensions.

“H-state” is a particular 3-D configuration or composition that thevarious 3-D objects take in H-space at a particular time. In otherwords, H-state is a 3-D snapshot of the system's health at one point oftime.

“Life-space” or “L-space” provides the present and past health states ofa system in a historical and comparative view of the evolution of thesystem in time. This 3-D representation environment constitutes thehistorical or Life-space of a dynamic system. L-space allows for bothcontinuous and categorical displays of temporal dependent complex data.In other words, L-space represents the health history or trajectory ofthe system in time.

“Real-Time Representation” is the display of a representation of thedata within a fraction of a second from the time when the event of themeasured data occurred in the dynamic system.

“Real-Time User Interface” is the seemingly instantaneous response inthe representation due to user interactivity (such as rotation andzooming).

A “variable” is a time dependent information unit (one unit per timeincrement) related to sensing a given and constant feature of thedynamic system.

“Vital signs” are key indicators that measure the system's criticalfunctions or physiology. In the preferred embodiments of this invention,data is gathered using methods or processes well known in the art or asappropriate and necessary. For example, in general, physiologic data,such as heart rate, respiration rate and volume, blood pressure, and thelike, is collected using the various sensors that measure the functionsof the natural system. Sensor-measured data is electronicallytransferred and translated into a digital data format to permit use bythe invention. This invention uses the received measured data to deliverreal-time and/or historical representations of the data and/or recordeddata for later replay. Moreover, this invention permits the monitoringof the health of a dynamic system in a distributed environment. Bydistributed environment, it is meant that a user or users interactingwith the monitoring system may be in separate locations from thelocation of the dynamic system being monitored. In its most basicelements, the monitoring system of this invention has three majorlogical components: (1) the sensors that measure the data of the system;(2) the networked computational information systems that computes therepresentation and that exchanges data with the sensors and the userinterface; and (3) the interactive user interface that displays thedesired representation and that interactively accepts the users' inputs.The components and devices that perform the three major functions ofthis invention may be multiple, may be in the same or different physicallocations, and/or may be assigned to a specific process or shared bymultiple processes.

FIG. 1 a is a top-level representative diagram showing the dataprocessing paths of the preferred embodiment of this invention operatingon a natural system. The natural system 101 a is shown as a dynamicentity whose origin, processes and structures (although not necessarilyits maintenance) were not manmade or artificially created. Examples ofnatural systems are living organisms, ecological systems, and variousEarth environments. In one preferred embodiment of the invention, ahuman being is the natural system whose physiology is being monitored.Attached to the natural system 101 a are a number of sensors 102. Thesesensors 102 collect the physiologic data, thereby measuring the selectedcritical functions of the natural system. Typically, the data gatheringof the sensors 102 is accomplished with methods or techniques well knownin the art. The sensors 102 are typically and preferably electricallyconnected to a digital data formatter 103. However, in other embodimentsof this invention, the sensors may be connected using alternative meansincluding but not limited to optical, RF and the like. In manyinstances, this digital data formatter 103 is a high-speed analog todigital converter. Also, connected to the digital data formatter 103 isthe simulator 101 b. The simulator 101 b is an apparatus or processdesigned to simulate the physiologic process underlying the life of thenatural system 101 a. A simulator 101 b is provided to generate vitalsign data in place of a natural system 101 a, for such purposes aseducation, research, system test, and calibration. The output of thedigital data formatter 103 is Real-Time data 104. Real-Time data 104 mayvary based on the natural system 101 a being monitored or the simulator101 b being used and can be selected to follow any desired time frame,for example time frames ranging from one-second periodic intervals, forthe refreshment rates of patients in surgery, to monthly statisticsreporting in an ecological system. The Real-Time data 104 is provided toa data recorder 105, which provides the means for recording data forlater review and analysis, and to a data modeling processor and process108. In the preferred embodiments of this invention the data recorder105 uses processor controlled digital memory, and the data modelingprocessor and process 108 is one or more digital computer devices, eachhaving a processor, memory, display, input and output devices and anetwork connection. The data recorder 105 provides the recorded data toa speed controller 106, which permits the user to speed-up or slow-downthe replay of recorded information. Scalar manipulations of the time(speed) in the context of the 3-D modeling of the dynamic recordeddigital data allows for new and improved methods or reviewing the healthof the systems 101 a,b. A customize/standardize function 107 is providedto permit the data modeling to be constructed and viewed in a widevariety of ways according to the user's needs or intentions.Customization 107 includes the ability to modify spatial scale, suchmodifying includes but is not limited to zooming, translating, androtating, attributes and viewports in addition to speed. In onepreferred embodiment of the invention, the range of customization 107permitted for monitoring natural systems 101 a physiologic states isreduced and is heavily standardized in order to ensure that data ispresented in a common format that leads to common interpretations amonga diverse set of users. The data modeling processor and process 108 usesthe prescribed design parameters, the standardized/customize functionand the received data to build a three-dimensional (3-D) model inreal-time and to deliver it to an attached display. The attached displayof the data modeling processor and process 108 presents a representation109 of 3-D objects in 3-D space in time to provide the visualrepresentation of the health of the natural system 101 a in time, or asin the described instances of the simulated 101 b system.

FIG. 1 b is a top-level block diagram of the data processing flow of thepreferred embodiment of this invention operating on an artificialsystem. An artificial system is a dynamic entity whose origin, processesand structure have been designed and constructed by human intention.Examples of artificial systems are manmade real or virtual, mechanical,electrical, chemical and/or organic entities. The artificial system 110a is shown attached to a number of sensors 111. These sensors 111collect the various desired data, thereby measuring the selectedcritical functions of the artificial system. Typically, the datagathering of the sensors 111 is accomplished with methods or techniqueswell known in the art. The sensors 111 are connected to a data formatter112, although alternative connection means including optical, RF and thelike may be substituted without departing from the concept of thisinvention. In many instances, this digital data formatter 112 is ahigh-speed analog to digital converter. Although, in certainapplications of the invention, namely stock market transactions, thedata is communicated initially by people making trades. Also connectedto the digital data formatter 112 is the simulator 110 b. The simulator110 b is an apparatus or process designed to simulate the processunderlying the state of the artificial system 110 a. The simulator 110 bis provided to generate vital data in place of the artificial system 110a, for such purposes as education, research, system test, andcalibration. The output of the digital data formatter 112 is Real-Timedata 113. Real-Time data 113 may vary based on the artificial system 110a being monitored or the simulator 110 b being used and can be selectedto follow any desired time frame, for example time frames ranging frommicrosecond periodic intervals, for the analysis of electronic systems,to daily statistics reported in an financial trading system. TheReal-Time data 113 is provided to a data recorder 114, which providesthe means for recording data for later review and analysis, and to adata modeling processor and process 117. In the preferred embodiments ofthis invention the data recorder 114 uses processor controlled digitalmemory, and the data modeling processor and process 117 is one or moredigital computer devices, each having a processor, memory, display,input and output devices and a network connection. The data recorder 114provides the recorded data to a speed controller 115, which permits theuser to speed-up or slow-down the replay of recorded information. Scalarmanipulations of the time (speed) in the context of the 3-D modeling ofthe dynamic recorded digital data allows for new and improved methods orreviewing the health of the system 110 a,b. A customize/standardizefunction 116 is provided to permit the data modeling to be constructedand viewed in a wide variety of ways according to the user's needs orintentions. Customization 116 includes the ability to modify spatialscale (such modification including, but not limited to translating,rotating, and zooming), attributes, other structural and symbolicparameters, and viewports in addition to speed. The range ofcustomization form monitoring artificial systems' 110 a,b states is wideand not as standardized as that used in the preferred embodiment of thenatural system 101 a,b monitoring. In this Free Customization, thesymbolic system and display method is fully adaptable to the user'sneeds and interests. Although this invention has a default visualizationspace, its rules, parameters, structure, time intervals, and overalldesign are completely customizable. This interface modecustomize/standardize function 116 also allows the user to select whatinformation to view and how to display the data. This interface modecustomization 116 may, in some preferred embodiments, producepersonalized displays that although they may be incomprehensible toother users, facilitate highly individual or competitive pursuits notlimited to standardized interpretations, and therefore permit a user tolook at data in a new manner. Such applications as analysis of stockmarket data or corporation health monitoring may be well suited to theflexibility of this interface mode. The data modeling processor andprocess 117 uses the prescribed design parameters, thecustomize/standardized function 116 and the received real-time data 113to build a three-dimensional (3-D) model in time and to deliver it to adisplay. The display of the data modeling processor and process 117presents a representation 118 of 3-D objects in 3-D space in time toprovide the visual representation of the health of the artificial system110 a in time, or as in the described instances of the simulated 110 bsystem.

FIG. 1 c is a top-level block diagram of one preferred processing pathof this invention. Sensors 119 collect the desired signals and transferthem as electrical impulses to the appropriate data creation apparatus120. The data creation apparatus 120 converts the received electricalimpulses into digital data. A data formatter 121 receives the digitaldata from the data creation apparatus 120 to provide appropriateformatted data for the data recorder 122. The data recorder 122 providesdigital storage of data for processing and display. A data processor 123receives the output from the data recorder 122. The data processor 123includes a data organizer 124 for formatting the received data forfurther processing. The data modeler 125 receives the data from the dataorganizer and prepares the models for representing to the user. Thecomputed models are received by the data representer 126, which formatsthe models for presentation on a computer display device. Receiving theformatted data from the data processor 123 are a number of datacommunication devices 127, 130. These devices 127, 130 include a centralprocessing unit, which controls the image provided to one or more localdisplays 128, 131. The local displays may be interfaced with a custominterface module 129 which provides user control of such attributes asspeed 131, object attributes 132, viewports 133, zoom 134 and other likeuser controls 135.

FIG. 1 d is a top-level block diagram of a second preferred processingpath of this invention. In this embodiment of the invention a pluralityof entities 136 a,b,c are attached to sensors 137 a,b,c whichcommunicate sensor data to a data collection mechanism 138, whichreceives and organizes the sensed data. The data collection mechanism138 is connected 139 to the data normalize and formatting process 140.The data normalize and formatting process 140 passes the normalized andformatted data 141 to the distributed processors 142. Typically andpreferably the processing 142 is distributed over the Internet, althoughalternative communication networks may be substituted without departingfrom the concept of this invention. Each processing unit 142 isconnected to any of the display devices 143 a,b,c and receives commandcontrol from a user from a number of interface units 144 a,b,c, each ofwhich may also be connected directly to a display devices 143 a,b,c. Theinterface units 144 a,b,c receive commands 145 from the user thatprovide speed, zoom and other visual attributes controls to the displays143 a,b,c.

FIGS. 2 a, 2 b, 2 c, and 2 d are representative 3-D objects representingcritical functions. Each 3-D object is provided as a symbol for acritical function of the entity whose health is being monitored. Thesymbol is created by selecting the interdependent variables that measurea particular physiologic function and expressing the variable in spatial(x,y,z) and other dimensions. Each 3-D object is built from 3-D objectprimitives (i.e., a cube, a sphere, a pyramid, a n-polygon prism, acylinder, a slab, etc.). More specifically, the spatial dimensions(extensions X, Y and Z) are modeled after the most important physiologicvariables based on (1) data interdependency relationships, (2) rate,type and magnitude of change in data flow, (3) geometric nature andperceptual potential of the 3-D object, for example a pyramid versus acylinder, (4) potential of the object's volume to be a data-variableitself by modeling appropriate data into x, y and z dimensions (e.g., inone preferred application of the invention, cardiac output is the resultof heart rate (x and y dimensions) and stroke volume (z)), (5)orthographic viewing potential (see viewport) and (6) the relationshipwith the normal values framework.

The first representative object 201, shown in FIG. 2 a, is an engineprocess. The object 201 representing this process is provided on astandard x-y-z coordinate axis 202. The correlation between temperature,shown in the x1-dimension 204, engine RPM, shown in the y1-dimension 205and exhaust gas volume, shown in the z1-dimension 203 is shown bychanges in the overall sizes and proportion of the object 201. In theshown example object 201 the engine gas volume 203 is large, when RPM205 is low and the engine temperature 204 is in the middle range. Thiscombination of values, even without specific identified values suggestsan engine's starting point.

The second representative object 206, shown in FIG. 2 b, is an objectrepresenting cardiac function using stroke volume, in the y2-dimension209, and the heart rate per second, shown as the x2, z2 dimensions. Thetotal cardiac volume is shown as the total spherical volume 208.

The third representative object 211, shown in FIG. 2 c, represents theinteraction between the number of contracts, shown in the y3-dimension212, the average revenue per contract, shown in the z3-dimension 214,and the average time per contract, shown in the x3-dimension 213.Assessing the interaction among these variables is important inmonitoring of a sales department's operations.

The fourth representative object 215 is shown in FIG. 2 d, shows therespiratory function generated by the respiratory rate, shown inx4-dimension 216, the respiratory volume, shown in the y4-dimension 216,and inhalation/exhalations, shown in the z4-dimension 218.

FIG. 3 is a representation of data objects in H-space 301. Data sets arerepresented as 3-D objects of various characteristics and relationshipswithin a 3-D representation space. The data representation environmentin this figure is used to map the physiologic data in 3-D and is what isreferred to as “Health-space” or “H-space”301. The 3-D objects areplaced within H-space on the 3 coordinates of their geometric centers.The coordinates for an object's geometric center depends on the relevantdata associated to the particular critical function the objectrepresents. For example, in the preferred embodiment, the cardiacfunction object, shown as a spherical object 302, is placed in H-space301 based on Mean Blood Pressure, designated as Oy 306 and OxygenSaturation in the Blood, shown as Oz 307. In the other example object,the prism 309 is placed in H-space 301 depending on sales profit, shownas Py 312, and products in stock, shown as Pz, 311. The location of 3-Dobjects in H-space 301 allows for the overall extension envelope ofH-space, the relationship between 3-D objects and spaces within H-space301, the viewport display areas and the departure from normative values.Typically and preferably the centers of the objects 302, 309 are locatedin the middle of the x-dimension of H-space 301.

FIGS. 4 a and 4 b are representative views of changes in data objects intime. In FIG. 4 a, the x-coordinate 400 is used to measure the temporaldimension of an objects 402 trajectory. The y-z plane 401 a determinesthe location of an object's geometric center within H-space. Increasesor decreases in data values associated with the coordinates of theobject's geometric center that make that object's location change intime as shown in path line 401 b. In this view, the object 402 ispresented in four different time intervals 403, 404, 405, 406, therebycreating a historical trajectory. The time intervals at which the object402 is shown are provided 407. In FIG. 4 b, increases in size andproportion are presented, 408, 409, 410, 411 providing an example ofchanges in values. The monitoring of these changes in time assists theuser establish and evaluate comparative relationships within and acrossH-states.

FIGS. 5 a, 5 b, 5 c, 5 d, 5 e, 5 f, 5 g and 5 h are representative viewsof properties of data objects provided in the preferred embodiment ofthis invention. In addition to the three x-y-z spatial dimensions usedfor value correlation and analysis, 3-D objects may present data valuestates by using other geometric, aesthetic, and aural attributes thatprovide for the mapping of more physiologic data. These figures showsome of the representative other geometric, aesthetic, and auralattributes supported for data presentation in this invention. FIG. 5 ashows changes in apparent volumetric density. A solid object 501 isshown in relation to a void object 502 and an intermediate state 503object. FIG. 5 b shows changes in apparent 3-D enclosure. An open object504, a closed object 505, and an intermediate state 506 is shown. FIG. 5c shows the apparent degree of formal deformation. A normal object 507,a distorted object 508, a transformed object 509, and a destroyed object510 are shown in comparison. FIG. 5 d shows secondary forms of theobjects. “Needles” 513 protruding through a standard object 512 incombination 511 is shown in comparison with a boundary 515 surrounding astandard object 514 and a bar 517 protruding into the original formobject 518 forming a new combination object 516 are shown providingadditional combination supported in this invention. FIG. 5 e shows thevarious degrees of opacity of the object's surface, showing an opaqueobject 519, a transparent object 520 and an intermediate state object521. FIG. 5 f shows the various degrees of texture supported by theobject display of this invention, including a textured object 522, asmooth object 523 and an intermediate textured object 524. FIG. 5 g isintended to represent various color hue possibilities supported forobjects in this invention. An object with color hue is represented 525next to a value hue object 526 and a saturation hue object 527 forrelative comparison. Naturally, in the actual display of this inventioncolors are used rather than simply the representation of color shown inFIG. 5 g. FIG. 5 h shows the atmospheric density of the representationspace possible in the display of objects in this invention. Anempty-clear space 528, a full-dark space 530 and an intermediate foggyspace 523 are shown with 3-D objects shown within the representativespace 529, 531, 533.

Aural properties supported in this invention include, but are notlimited to pitch, timbre, tone and the like.

FIG. 6 shows the 3-D configuration of the objects in H-space in thepreferred embodiment of the invention. In this view the local level,H-space 601 is shown within which the 3-D objects 602, 603, and 604 arelocated. Object 602 represents the respiratory function of anindividual. Its 602 x-y-z dimensions change based on the parameter-baseddimensional correlation. The object 603 represents the efficiency of thecardiac system by varying the x,y,z coordinates of the object. Theobject 604 represents a human brain function, also with the x,y,zdimensions changing based on the parameter-based dimensionalcorrelation. In this way the user can easily view the relativerelationships between the three physiological objects 602, 603, 604.Within H-space 601, the temporal coordinate (i.e., periodic timeinterval for data capturing that defines how H-space is plotted inLive-space see FIG. 7) is a spatial dimension on which data is mapped.The x-dimension of 605 of the H-space 601 can be mapped to anotherindependent variable such as heart rate period, blood pressure or thelike. The location of an object in the y-dimension 606 of H-space 601can be mapped to additional variables that are desired to be monitoredsuch as SaO2 content, CaO2 content, or temperature in the blood. Thelocation of an object in the z-dimension 607 of the H-space 601 can alsobe mapped to additional variables that the user desires to monitor. Ahypothetical object 608 shows that the three coordinates are contextualto a particular object 608 and need not be the same for all objects,except in the object's 608 extension measuring properties. Fixed x- andz-dimension values 609 a and 609 b are shown as constant. The y-value610 of this object 608 changes to fluctuating values or data type thatresults in the height of the object 608 increasing or decreasing. Thisview shows another object 611 showing the relationship between the threedimensions. Constant x- and y-values 612 a and 612 b are shown. Thez-value 613 of this object 611 changes to fluctuating values or datatypes that result in the width of the object 611 increasing ordecreasing. An overlapping view 614 of an object 615 that has extendedpast the H-space limitation. A limit of H-space 616 with a sphericalobject 617 located inside H-space 616 shown with the degree of extensionshown in shaded circles.

FIG. 7 shows a series of H-spaces 701, 702, 703, 704, 705, 706 along aglobal time coordinate 708, and the local-space coordinates 707 thatgoverns each H-space. Each of these H-spaces represents progressivestates of the dynamic system at pre-established temporal intervals (T₀,T⁻¹, T⁻², . . . T_(−n)) and the six 701, 702, 703, 704, 705, 706together show the evolution of that system over time, demonstrating thehistorical representation of individual H-states within an overall“Life-space” or “L-space.” At the global level (or L-space), one of thecoordinates, typically x, is always time. The temporal coordinate isscaled based on the intervals at which a particular functions system'sphysiologic data are collected by the art or as appropriate. Thisinterval or module is fixed and constant across L-space and provides thenecessary temporal frame of reference for comparing different H-spaces.The fixed temporal interval also determines the maximum x-extension ofthe representation envelope of H-space. The other two coordinates, y andz, provide L-space with extension and are not fixed. The threecoordinates thus described provide a regulating 3-D environment withinwhich the H-states can be visualized and related to each other.

FIGS. 8 a and 8 b show the global level coordinate system of thepreferred embodiment of this invention. FIG. 8 a shows the L-spacecoordinate system 801 in its preferred embodiment. The x-dimension 802of L-space is mapped to a constant time interval, set by means standardin the art or otherwise as appropriate. The present position of H-stateis also indicated on the x-dimension 802. The y-dimension 803 in bothpositive and negative extensions is measured, up and down from thex-axis. This dimension 803 can be mapped to a data variable withinparticular 3D object in space. The z-dimension 804 is shown in bothpositive and negative extensions measured forwards and backwards fromthe intersecting x-axis. This dimension 804 can be mapped to a datavariable within a particular 3D object in space. Now for FIG. 8 b aprismatic object 800 represents a critical function, whose evolution isbeing monitored in L-space, of a given dynamic system. The front view805 shows the different H-states of the prism/function 800 using a timeT to T−n historical trend. The level of intersection and separationbetween the front views of the prism indicate abnormal health states ofthe critical function the object 800 represents. No separation orintersection shows normal function conditions. The trajectory in they-dimension of the prism (i.e., H-states of the critical function) aremapped to a variable that cause their relative position to change inthe + and y dimension. The current state 806 of the prism is shown inthis front view 805. A top view of 809 of the three-dimensional L-spaceis shown, showing the evolution of the prism 800 backward in time andshowing a T to T-N historical trend. The level of intersection andseparation indicate abnormal health states of the particular criticalfunction the prism represents. No separation or intersection showsnormal conditions. The trajectory in the z-dimension of the object ismapped to a variable that causes their position to change in the + and zdimension. This top view shows both the z and y trajectories in onecomprehensive view. The perspective view 808 of L-space gives acomprehensive view of the interaction of the prisms (the H-states of thefunction) and their movement in all dimensions. The side view 807 ofL-space shows the prisms and their positions in L-space giving asimultaneous view of z and y trajectories.

FIGS. 9 a and 9 b shows various viewpoints in which the data may bevisualized in the preferred embodiment of this invention. This figureshows representations of a data object (a prism) and is provided to showthat there are two basic types of viewports: orthographic andperspectival. The orthographic viewports 906, 907, 908, of FIG. 9 b usea parallel system of projection to generate representations of H-spacethat maintains dimensional constancy without deformation. Some examplesof orthographic views include traditional architectural or engineeringviews of objects, such as a top view, a front view, and a side view. Theorthographic viewport allows for accurate and focused 2-D expressions ofthe actual 3-D object. The perspectival viewport 909, shown in FIG. 9 buses a focal system of projection to generate depictions analogous toour perception of reality but at the cost of deformation and lack ofdimensional constancy. For example, the top view 902 along with the sideview 903 and the front view of 904 of the 3-D data object 901 are shownin FIG. 9 a. FIG. 9 b shows three orthogonal views 906, 907, 908 alongwith a perspective view 909 of the current data object. The number andtypes of viewports used in a particular embodiment of the invention mayrange from one type, for example a perspective viewport allowing immersevirtual reality, to combinations of both types. In the preferred currentembodiment, there are the four viewports shown in FIG. 9 b. Given the3-D nature of data objects and H-space, viewports provide the user withdifferent depictions of the same data.

FIG. 10 shows the transform of an object in space in context, with areference framework, in the preferred embodiment of this invention. Thereferential framework 1010 is typically set based on population normalsor patient normals. This framework assists the user to see deviationsfrom normal very quickly. An individual spherical object 1011 thatrepresents cardiac function is shown located in L-space and in relationto the referential framework. A side view 1012 is shown along withseveral cardiac objects. In this view the referential framework providesa center target point so that a user can make the necessary correctionsto bring the object back to the ideal center of the framework. Aperspectival view 1013 of the framework is also shown along with severalcardiac objects. The top view 1014 of the framework is shown withseveral spherical objects (representing cardiac states). This figuredemonstrates the variety of viewports provided to the user by thisinvention, which provides enhanced flexibility of analysis of thedisplayed data.

FIG. 11 a shows the zooming out function in the invention. Thisinvention provides a variety of data display functions. This figureshows the way views may be zoomed in and out providing the relativeexpansion or compression of the time coordinate. Zooming out 1101permits the user to look at the evolution of the system's health as itimplies the relative diminution of H-states and the expansion ofL-space. This view 1101 shows a zoomed out view of the front viewshowing a historical view of many health states. A side view 1102 zoomedout view is provided to show the historical trend stacking up behind thecurrent view. A 3-D perspectival, zoomed out view 1103 showing theinteraction of H-states over a significant amount of time is provided. Azoomed out top view 1104 shows the interaction of H-states over a largeamount of time.

FIG. 11 b shows the zooming in function of the invention. The zooming infront view 1105 is shown providing an example of how zooming in permitsa user to focus in on one or a few H-states to closely study specificdata to determine with precision to the forces acting on a particularH-state. A zoomed in side view 1106 is provided showing the details ofspecific variables and their interactions. A zoomed in 3-D perspectiveview 1107 of a few objects is also shown. Also shown is a zoomed in topview 1108 showing the details of specific variables and theirinteraction.

FIG. 12 a shows a 3-D referential framework of normative values that isprovided to permit the user a direct comparison between existing andnormative health states, thereby allowing rapid detection of abnormalstates. The reference framework 1201 works at both the global L-spacelevel and the local H-space level. “Normal” values are established basedon average historical behavior of a wide population of systems similarto the one whose health is being monitored. This normal valueconstitutes the initial or by-default ideal value, which, if necessarymay be adjusted to acknowledge the particular characteristics of aspecific system or to follow user-determined specifications. The highestnormal value of vital sign “A”1202 (+y) is shown, along with the lowestnormal value of “B”1203 (−z), the lowest normal value of vital sign“A”1204 (−y) and the highest normal value of vital sign “B” 1205 (+z).In FIG. 12 b, abnormal values of “A” and “B” are shown in an orthogonalview. An abnormally high value of “A”1206, an abnormally low value of“B”1207, an abnormally low value of “A” 1208 and an abnormally highvalue of “B”1209 are shown.

FIG. 13 shows a comparison of the interface modes of the preferredembodiment of this invention. This invention provides two basic types ofinterface modes: (a) standardized or constrained customization; and (b)free or total customization. Each is directed toward different types ofapplications. The standardized or constrained customization 1301 uses amethod and apparatus for user interface that is set a-priori by thedesigner and allows little customization. This interface modeestablishes a stable, common, and standard symbolic system anddisplaying method that is “user-resistant”. The fundamental rules,parameters, structure, time intervals, and overall design of L-space andH-space are not customizable. Such a normalized symbolic organizationcreates a common interpretative ground upon which different users mayarrive at similar conclusions when provided common or similar healthconditions. This is provided because similar data flows will generatesimilar visualization patterns within a standardized symbolic system.This interface method is intended for social disciplines, such asmedicine in which common and agreeable interpretations of the data arehighly sought after to ensure appropriate and verifiable monitoring,diagnosis and treatment of health states. The customization permitted inthis mode is minimal and is never threatening to render the monitoringdevice incomprehensible to other users.

The free or total customization interface mode 1302 provides a symbolicsystem and displaying method that is changeable according to the user'sindividual needs and interests. Although the invention comes with adefault symbolic L-space and H-space, its rules, parameters, structure,time intervals, and overall design are customizable. This interface modealso permits the user to select what information the user wishes to viewas well as how the user wishes to display it. This interface mode mayproduce personalized displays that are incomprehensible to other users,but provides flexibility that is highly desired in individual orcompetitive pursuits that do not require agreeable or verifiableinterpretations. Examples of appropriate applications may include thestock market and corporate health data monitoring.

FIG. 14 is a hardware system flow diagram showing various hardwarecomponents of the preferred embodiments of the invention in a “naturalsystem” medical application. Initially a decision 1401 is made as to theoption of using data monitored on a “real” system, that is a realpatient, or data from the simulator, for anesthesiology trainingpurposes. If the data is from a real patient, then the patient 1402 isprovided with patient sensors 1404, which are used to collectphysiological data. Various types of sensors, including but not limitedto non-invasive BP sensors, ECG leads, SaO2 sensors and the like may beused. Digital sensors 1416 may also provide physiological data. An A/Dconverter 1405, is provided in the interface box, which receives theanalog sensor signals and outputs digital data to a traditional patientmonitor 1406. If the data is produced 1401 by the simulator 1403, acontrol box and mannequins are used. The control box controls thescenarios simulated and the setup values of each physiological variable.The mannequins generate the physiological data that simulates realpatient data and doctors collect the data through different, butcomparable sensors. The traditional patient monitor 1406 displays thephysiological data from the interface box on the screen. Typically andpreferably, this monitor 1406 is the monitor used generally in an ICU. Atest 1407 is made to determine the option of where the computations anduser interface are made, that is whether they are made on the networkserver 1408 or otherwise. If a network server 1408 is used, all or partof the data collection and computation may be performed on this computerserver 1408. An option 1409 is proved for running a real timerepresentation versus a representation delayed or replayed from eventsthat previously occurred. For real time operation, a data buffer 1410 isprovided to cache the data so that the representation is played in realtime. For the replay of previous events, a data file 1411 provides themeans for permanently storing the data so that visualization isreplayed. The visualization software 1412 runs on a personal computerand can display on its monitor or on remote displays via the internet orother networking mechanism. Typically the physiological data measured oneither a real patient or the simulator are fed to the personal computerfrom the traditional data monitor. A standard interface such as RS232,the Internet, or via a server, which receives data from the monitor, mayserve as the communication channel to the personal computer running thevisualization software 1412. This program 1412 is the heart of theinvention. The program 1412 computes the representation and processesthe user interface. An option 1413 is provided for computing and userinterface on the local desktop personal computer or for distributionacross the Internet or other network mechanism. If a local desktoppersonal computer is selected, the personal computer 1414 with anadequate display for computation of the visualization and user interfaceis provided. If a remote user interface 1415 is selected the display anduser interface is communicated across the Internet.

FIG. 15 is a software flow chart showing the logic steps of a preferredembodiment of the invention. The preferred embodiment of this inventionbegins by reading the startup file 1501, which contains the name of thewindow and the properties associated with the invention. The propertiesassociated with the a window include formulas to set object properties,text that is to be rendered in the scene, the initial size of thewindow, the initial rotation in each window, zoom, lighting and patientdata that describes the normal state of each variable. Internal datatables are next initialized 1502. For each new window encountered in thestartup file a new window object is made and this window object isappended to the list of windows. The window object contains anuninitialized list of properties describing the state of the window,which is filled with data from the startup file. The event loop isentered 1503. This is a window system dependent infinite loop from whichthe program does not exit. After some initialization, the program waitsfor user input and then acts on this input. The program then takescontrol of the event loop for continuous rendering that is if there isno interactivity in the program. Initialization 1504 of windows is nextperformed. This involves calls to the window system dependent functions(these are functions that are usually different on differentcomputational platforms) that creates the windows and displays them onthe computer screen. In the current preferred embodiment of theinvention, OpenGL is required, although alternative embodiments usingother 3D application programming interfaces, such as PEX or DirectX,could be substituted without departing from the concept of thisinvention. Also, in the preferred embodiment of this invention, apersonal computer graphics card is preferred in the personal computer soas to permit smooth animation with multiple windows. Although the lackof such a card is not absolutely required for operation of thisinvention. New data is received 1509, typically from the data file 1506or the data buffer 1507. This new data 1509 can come from any sourcethat generates floating-point numbers. The preferred line of data iscomposed of columns of floating point numbers separated by space. Atthis point the current time is also stored so that the next line of datacan be obtained at the next user defined time interval, which istypically set at about 1 second. Object properties are next computed1510. This is performed by using formulas that are specified in thestartup file to compute properties of objects. Data fields in theformulas are specified by writing the column number preceded by a dollarsign. For example, $1/20.0 would divide the first field by 20.0. Thespecific properties in this application are: cardiac object dimensions,material properties, and position. Material properties can include thered, green, and blue components as they appear under ambient, diffuse,and specular light, as well as transparency. The cardiac object positionincludes the y and z positions as well as an x shift. If four or morelines of data have been acquired, the respiratory object properties arecomputed. A delay is necessary because a cubic spline is fitted, usingfour data points to do the fit, to the data points to generate a smoothrespiratory object. Therefore, until four time steps have passed, thecurtain is not rendered. Thereafter, it is rendered every time new datais acquired. Cardiac object properties include material properties andthe height of the color bands. Blood pressure object length andmaterials are the thin cylinders that go through the top and bottom ofeach ellipsoid. Next, reference grid properties are computed. Allobjects, except the cardiac object reference are stationary, in thecurrent implementation. The cardiac object reference can move accordingto the movement of the cardiac object if the user specifies thismovement in the startup file. Next, sounds are computed 1511 and madeaudible 1513. Objects and reference grids are rendered 1512. Beforerotation the newest object appears at the right side of the screen andoldest object is at the left side of the screen. Sound is produced 1513next. A test 1514 is next made to determine if smooth animation isselected. If smooth animation is selected the scene will scroll duringthe time the program is waiting to get new data. The program, usingavailable computing resources, selects the minimum time increment sothat the shift of the objects can be rendered within the increment, butlimiting the increment to the smallest increment that human eyes candetect. If smooth animation is not selected, objects are shifted to theleft 1515 such that the distance from the center of the newest cardiacobject to that of the former cardiac object is equal to theinter-cardiac spacing. The process waits 1508 until the current timeminus the time since data was last obtained equals the data acquisitionperiod specified by the user. If the current time minus the time whenthe data was last acquired equals the user specified data acquisitionperiod then a new line of data is acquired. If smooth animation isselected, then the cardiac objects are shifted to the left by computing1516 to that when it is time to get the next line of data, the cardiacobjects have moved 1517, 1518 such that the distance from the rightmostcardiac object to the position where the new cardiac object will appearis equal to the inter-cardiac-object distance. For example, if it takes0.20 seconds to render the previous scene, the period of dataacquisition is 1.0 seconds, and the x shift of the rightmost cardiacobject is 0.1 units then the program will shift the scene left(0.20/(1.0+0.20)*(1.0 0.1)=0.15. The formula in the denominator is(1.0+0.20 instead of 0.8 because, if the scene has been shifted leftsuch that, when new data is acquired, the shifting has stopped (becausethe position of the cardiac objects satisfies the criteria that thedistance from the center of the rightmost cardiac object to the centerpoint where the new cardiac object will be rendered=1 unit) then theanimation will no longer be smooth, that is, when new data is acquiredthe animation will appear to stop. Note, that the respiratory object isnever entirely smoothly shifted because no data is available to renderthe object at the intermediate time steps.

FIG. 16 is a software block diagram showing the logic steps of the imagecomputation and rendering process of a preferred embodiment of theinvention. This process begins with acquiring the window identification1601 of the current rendering context. Next, the data structure is found1602 corresponding to the current window identification. After which,the view is set 1603. A rotation matrix is set 1604. A projection matrixis set 1605. Lights are set 1606. The back buffer is cleared 1607.Object processing 1608 begins, and includes for each cardiac object,calling OpenGL to see material properties; shift left oneinter-cardiac-object distance; push the modelview matrix, shift x,y, andz directions; call OpenGL utility toolkit to render the cardiac object;set the top cardiac object material properties, call OpenGL quadriesfunction to render top cardiac object; set top cardiac object materialproperties, call OpenGL quadrics function to render bottom cardiacobject and pop modelview matrix. Next, the view is set 1609, as above.The respiratory object is rendered 1610, by setting OpenGL to renderquad strips, for each polygon strip set material properties, and sendvertex to OpenGL. Reference grids are rendered 1611 by setting materialproperty of the cardiac reference grid. The current position is set 1612to be the ideal position of the newest cardiac object, that is theposition corresponding to a patient in ideal health. The cardiac objectmaterial properties are set 1613. The OpenGL utility toolkit is calledto render 1614 the cardiac object. Next, OpenGL is set to render quads1615. After which the material properties of the reference planes areset 1616. Vertices that compose the reference planes through the OpenGLpipeline are sent 1617 and buffers are swapped 1618. Buffer swap is awindow system defendant function.

FIG. 17 is a photograph of the 3-dimensional display of a preferredembodiment of the invention. The 3-D view shown at lower right 1706provides a comprehensive, integrated and interactive view of allphysiological data, and shows the interaction between the differentobjects in relation to the reference frame. This view can be manipulatedby the user to fit specific application needs. The front 1701, side1704, 1705 and top views 1702 show how the same data appears fromdifferent vantage points. In this figure these views 1701, 1702, 1704,1705 show the interaction between the cardiac object, the referenceframe and the respiratory object, with the side view 1704 providing atarget for optimum efficiency of the cardiac system 1705 shows the levelof gas concentration in the lungs and overall tidal volume in therespiratory system. This FIG. 17 is a representation of a true 3-D modelof the physiologic data. The circle 1703 shown is the top view of therespiratory waveform showing CO2 content in the lungs and inspirationand expiration values. In 1703, a real time display, the object growsand shrinks with each heartbeat. Its height is proportional to theheart's volume output and its width is proportional to heart rate. Thegridframe (or reference framework) shows the expected normal values forstroke volume and heart rate. The position of this object in thevertical direction of the display is proportional to the patient's meanblood pressure. This graphic objects shape and animation provides auseful graphical similarity to a working heart. In the preferredembodiment, the background is colored to show inspired and expiredgases. The height of the “curtain” is proportional to tidal volume,while the width is proportional to respiratory rate. The colors, whichare, displayed in the preferred display show the concentrations ofrespiratory gases. Time is set to move from right to left, with thepresent or current conditions shown at the “front” or right edge of eachview. Past states remain to provide a historical view of the data.

FIG. 18 is a close-up front view of the cardiac object and theassociated reference framework of a preferred embodiment of theinvention. The upper limit of normal blood pressure value is shown 1800on the reference frame. The systolic blood pressure level is indicatedby the bar 1801 penetrating the cardiac sphere 1806. The height 1802 ofthe sphere 1806 is proportional to cardiac output, which shows theoptimum efficiency of the heart. The width of the sphere 1806 isproportional to 1/heart rate. The elevation of the sphere 1806 is anindication of mean blood pressure, where the center reference gridlineis a normal mean blood pressure 1803. The lower limit, or diastolicblood pressure 1804 is shown by the length of the bar extending downwardfrom the sphere 1806. Previous historical values for the sphere 1806 arealso provided in 1805, 1807.

FIG. 19 is a view of the front view portion of the display of apreferred embodiment of the present invention showing the cardiac objectin the foreground and the respiratory object in the background. Thisview 1900 provides a more quantitative image of the hemodynamicvariables, stroke volume, blood pressure 1901 and heart rate. The“normal” reference lines are more apparent. In the preferred embodiment,respiration is shown by changes in the background color.

FIG. 20 is a view of the top view portion of the display 2000 of apreferred embodiment of the present invention showing the cardiac objecttoward the bottom of the view and the respiratory object toward the topof the view. Inhaled gas 2002 and exhaled gas 2003. CO2 concentrationsand oxygen saturation of the arterial blood 2001 versus time are alsoshown.

FIG. 21 is a view of the side view portion of the display of a preferredembodiment of the present invention showing the cardiac object to theleft and the respiratory object to the right. Gas concentration in thelungs 2101, a calibrated scale for gas concentration 2103, bloodpressure 2100, and oxygen saturation 2101 are shown. The end view, shownhere in FIG. 21, is especially useful during treatment, where the goalis to bring the variables back to the center or normal state. Functionalrelationships can be added to this view to predict how treatment can beexpected to bring the variables back to normal.

FIG. 22 is a view of the 3-D perspective view portion of the display ofa preferred embodiment of the present invention showing the cardiacobject in the left foreground and the respiratory object in the rightbackground. This view 2200 provides a comprehensive, integrated andinteractive view of nine physiological variables. The sphere 2201 growsand shrinks with each heartbeat. Its height is proportional to theheart's stroke volume and its width is proportional to heart rate. Thisgraphic object 2201 offers useful similarity to a beating heart. Thegridframe 2202 shows the expected normal values for stroke volume andheart rate. The position of this object 2201 on the screen isproportional to the patient's mean blood pressure. The ends of the bar2203 drawn vertically through the center of the heart object showsystolic and diastolic blood pressure. In the preferred embodiment ofthe invention, the background 2204 is colored to show inspired andexpired gases. The height of the “curtain” 2205 is proportional to tidalvolume. The width of each fold 2206 is proportional to respiratory rate.In the preferred embodiment colors are used to show the concentrationsof respiratory gases. Time moves from right to left with the presentcondition shown at the “front” or right edge of the view 2200. Paststates 2207 remain to permit a historical view of the data.

FIG. 23 a shows the preferred graphic element of this inventiondepicting a normal cardiovascular system. This graphic element 2300 iscomposed of a number of distinct objects 2301, 2301, 2303, 2304, 2305,2306. Normal, or expected object represented values are shown by thefilling of an object in its designated frame 2301, 2301 a, 2303 a, 2304a, 2305 a, 2306 a. Numeric values 2307 a-e are also shown to providenumeric indications of the desired graphic object. Although shown hereas black objects within a white frame, in alternative embodiments theobjects and frames may be any desired displayable color, texture,shading and the like.

FIG. 23 b shows the preferred graphic element of this inventiondepicting a cardiovascular system exhibiting anaphylaxis. This figuredemonstrates the display of objects 2308, 2309, 2312 having valuessubstantially less than desired or expected, by failing to fill theexpected frame 2308 a, 2309 a, 2312 a. An object 2313 having a valuemuch larger than desired or expected is shown by overfilling its frame2313 a. Objects 2310, 2311 having expected values is shown by fillingtheir respective frames 2310 a, 2311 a. Two sloped regions 2314, 2315are provided to show a change in value between two objects.

FIG. 23 c shows the preferred graphic element of this inventiondepicting a cardiovascular system exhibiting hypovolemia. This figuredemonstrates the display of objects 2316, 2317, 2318, 2319, 2320, 2321having values substantially less than desired or expected, by failing tofill the expected frame 2316 a, 2317 a, 2318 a, 2319 a, 2320 a, 2321 a.Three sloped regions 2322, 2323, 2324 are provided to show a change invalue between two objects.

FIG. 23 d shows the preferred graphic element of this inventiondepicting a cardiovascular system exhibiting bradycardia. This figuredemonstrates the display of objects 2329, 2330 having valuessubstantially less than desired or expected, by failing to fill theexpected frame 2329 a, 2330 a. Objects 2326, 2327 having a value muchlarger than desired or expected is shown by overfilling its frame 2326a, 2327 a. And an object 2325 having an expected value is shown byfilling its respective frame 2325 a. Three sloped regions 2331, 2332,2333 are provided to show a change in value between two objects.

FIG. 23 e shows the preferred graphic element of this inventiondepicting a cardiovascular system exhibiting ischema. This figuredemonstrates the display of objects 2338, 2339 having valuessubstantially less than desired or expected, by failing to fill theexpected frame 2338 a, 2339 a. An object 2336 having a value much largerthan desired or expected is shown by overfilling its frame 2336 a.Objects 2334, 2335 having expected values is shown by filling theirrespective frames 2334 a, 2335 a. Two sloped regions 2340, 2341 areprovided to show a change in value between two objects.

FIG. 23 f shows the preferred graphic element of this inventiondepicting a cardiovascular system exhibiting pulmonary embolism. Thisfigure demonstrates the display of objects 2343, 2344, 2345, 2346, 2347having values substantially less than desired or expected, by failing tofill the expected frame 2343 a, 2344 a, 2345 a, 2346 a, 2347 a. Anobject 2342 having a value much larger than desired or expected is shownby overfilling its frame 2342 a. Two sloped regions 2348, 2349 areprovided to show a change in value between two objects.

FIG. 24 shows the preferred reference grid of this embodiment of thisinvention. A reference grid 2400 is provided within which space isallocated for graphic objects 2401, 2402, 2403, 2404.

FIG. 25 shows the preferred reference grid 2400 of this embodiment ofthis invention with the preferred object placement 2501, 2502, 2503,2504 as well as a center line axis point 2500 indicated. Generally thecenter line axis 2500 is used to scale the object from a center point.Smaller objects, such as 2502, indicates lower values. While largerobjects, such as 2503, indicates larger values.

FIG. 26 shows the preferred reference frame 2400 of this embodiment ofthis invention with the preferred object relationships indicated. Thisfigure provides the placement of objects identified with specificphysical processes. For example, object 2601 is CVP, object 2602 is PAP,object 2603 is LAP, object 2604 is HR SV and object 2605 is MAP. Inalternative applications of this invention, different physical processescan be assigned to the objects.

FIG. 27 shows a representative three-dimensional object 2700 used in thepresent preferred embodiment of this invention. This preferred object2700 is generally cylindrical in shape placed in a three-dimensionalcoordinate system 2701. Alternative object shapes are foreseeable andcan be substituted without departing from the concept of this invention.

FIG. 28 shows a representative view of the normalization of the presentpreferred embodiment of this invention. This figure shows thenormalization of frames 2801 a-e set within a reference grid 2800.

FIG. 29 shows and integrated view presenting numeric information 2901 inthe present preferred embodiment of this invention set to associate withparticular objects 2902 a-d within a reference frame 2900.

FIG. 30 shows the addition of slope features 3004, 3005 to show therestriction of blood flow, designated in objects 3001, 3002, 3003,within a reference frame 3000.

In one embodiment of the invention, a pulmonary metaphor was developedto improve the anesthesiologist's situational awareness. The metaphordisplays the patient's physiological status in an intuitive andmeaningful way by depicting metaphor signatures representing criticalpulmonary variables and events. Pulmonary variables include tidalvolume, respiratory rate, fractional inspired oxygen, peak endexpiratory pressure, oxygen saturation in arterial blood, oxygen partialpressure in arterial blood, carbon dioxide partial pressure in arterialblood, oxygen partial pressure in alveoli, carbon dioxide partialpressure in alveoli, end tidal carbon dioxide, functional residualcapacity, acidity of blood, peak inspiratory pressure, dead space,compliance, airway resistance, ventilation/perfusion, shunting of blood,and alveoli/arterial oxygen gradient. Pulmonary events includehypoventilation, hyperventilation, COPD, intrinsic PEEP, ventilator off,stiff lung, pneumothorax, bronchospasm, obstructed ETT, esophagealintubation, hypoxemia, endobronchial intubation, and hypercarbia.

FIG. 31 shows a an embodiment of a pulmonary metaphor 3100 thatanatomically represents the bellows 3101, airway 3102, lungs 3103,inspired gas 3104, and expired gas 3105. In one embodiment the bellows3101 was chosen to be a gray/blue color similar to the bellows in aventilator. The bellows 3101 move along the y-axis 3106 representingtidal volume. The bellows 3101 bulges along the x-axis 3107 representingPeak Inspiratory Pressure (PIP). The airway 3102 was chosen to besilver, and changes along the x-axis 3107 respective to airwayresistance. An obstructed airway would be depicted as a roundednarrowing of the tube. The lung 3103 could also be considered thealveoli or site of gas exchange interchangeably. The shade of green maybe used to map the FO2 present in the alveoli 3103. The black line 3108surrounding the lung 3103 is mapped inversely to compliance. The thickerthe line; the lower the compliance; the stiffer the lung. The width ofthe lung 3103 is mapped to intrinsic Peak End Expiratory Pressure(PEEP). As the intrinsic PEEP increases, the lung 3103 bulges out on itssides depicting a high internal pressure as shown in FIG. 33.Inhaled/inspired gas 3104 may be green in color and is mapped tofractional inspired oxygen (FIO2). Green is a common color used onphysiological monitors for oxygen. Exhaled/expired gas 3105 may becolored gray and is mapped to end tidal carbon dioxide (ETCO2). Gray isa common color used on physiological monitors for carbon dioxide. Areference frame 3109, 3110, 3111, 3112, 3113, surrounds each individualfeature. An abnormal value change is depicted in the positive ornegative direction from the reference frame 3109, 3110, 3111, 3112,3113. Subsequently, the abnormal emerging features will appear to popout and the metaphor will no longer look symmetrical.

FIG. 32 shows an embodiment of a pulmonary metaphor where variousfeatures are abnormal. The reference frames of each feature 3201, 3202,3203, 3204, 3205 are outlined in black. The bellows 3206 havecompressed, the airway 3207 has constricted, and expired gas 3208 hasincreased.

FIG. 33 shows an embodiment of a pulmonary metaphor where the intrinsicPEEP has increased. This has caused the lung 3301 to bulge out on itssides bond the normal reference frame 3302 to depict a high internalpressure. Additionally, the expired gas 3303 has decreased, and theairway 3304 has constricted.

FIG. 34 shows a preferred embodiment of a pulmonary metaphor. Thebellows 3401 are connected the lungs 3405 via the airway 3404. Inspiredgas 3403 and expired gas 3402 are located on either side of the bellows3401.

FIG. 35 shows an embodiment of a pulmonary metaphor wherein the patienthas pulmonary fibrosis. The bellows 3501 has bulged out the sides andconstricted downward. The airway 3502 shows resistance and the expiredgas 3503 has increased.

FIG. 36 shows an alternative embodiment of a pulmonary metaphor whereinthe patient has pulmonary fibrosis. The bellows 3601 has bulged out thesides and constricted downward. The airway 3602 shows resistance and theexpired gas 3603 has increased.

FIG. 37 shows an embodiment of a pulmonary metaphor wherein the patienthas emphysema. The airway 3701 is restricted and lungs 3703 are bulgingand the expired gas 3702 has decreased.

FIG. 38 shows an alternative embodiment of a pulmonary metaphor whereinthe patient has emphysema. The airway 3801 is restricted and lungs 3803are bulging and the expired gas 3802 has decreased.

It is to be understood that the above-described embodiments and examplesare merely illustrative of numerous and varied other embodiments andapplications which may constitute applications of the principles of theinvention. Such other embodiments may be readily devised by thoseskilled in the art without departing from the spirit or scope of thisinvention and it is our intent that they are deemed to be within thescope of this invention.

1. A medical display method for using a data processor to provide anintegrated representation of a patient's pulmonary system for diagnosticpurposes and display to a user, comprising: displaying an inspired gasobject on a display, wherein the inspired gas object is configured todepict a patient's inspired gas volume as measured by a first sensorconfigured to detect inspired gas; displaying a lung object on thedisplay, the lung object being associated with the inspired gas objectand configured to represent a patients' lung condition as sensed by asecond sensor configured to detect a lung condition; displaying anexpired gas object on the display, the expired gas object beingconfigured to represent a patient's expiration volume as measured bythird sensor configured to detect expired gas, and the expiration objectbeing displayed in combination with the lung object and inspired gasobject to exhibit interaction between objects to display pulmonarystates of the patient; and displaying a normal reference frame on thedisplay, the normal reference frame representing expected normalpulmonary values for inspired gas, lungs and expired gas.
 2. A medicaldiagnostic display method as in claim 1, further comprising: displayingan airway object representing a constriction state of a patient'sairway; displaying a ventilation object in communication with the airwayobject representing tidal volume of the patient's pulmonary system.
 3. Amedical diagnostic display method as in claim 2, wherein a ventilationobject is configured in a symbolic lung shape.
 4. A medical diagnosticdisplay method as in claim 1, wherein the lung object represents alveolior a site of gas interchange.
 5. A medical diagnostic display method asin claim 1, further comprising displaying a lung reference frame havinglung reference frame thickness mapped inversely to lung compliance,wherein a thicker lung reference frame represents increased lungstiffness.
 6. A medical diagnostic display method as in claim 5, whereinthe lung object is mapped to Positive End Expiratory Pressure (PEEP). 7.A medical diagnostic display method as in claim 5, wherein the lungobject can expand in width past the lung reference frame to representhigh internal lung pressure or volume.
 8. A medical diagnostic displaymethod as in claim 1, wherein changes on an x-axis of the airway objectrepresent Positive Inspiratory Pressure (PIP).
 9. A medical diagnosticdisplay method as in claim 1, wherein the inspired gas object isconfigured to represent fractional inspired oxygen concentration (FIO2).10. A medical diagnostic display method as in claim 1, wherein theexpired gas object is configured to represent end tidal carbon dioxideconcentration (ETCO2).
 11. A medical diagnostic display method as inclaim 1, wherein the normal reference frame and object within thereference frame are configured to show positive increases pastnormalization by expanding outside the reference frame and negativedecreases from normalization by shrinking within the reference frame.12. A medical diagnostic display method for using a data processor toprovide an integrated view of a patient's pulmonary system for displayto a user, comprising: displaying an inspired gas object configured todepict a patient's inspired gas volume as measured by a first sensorconfigured to detect inspired gas; displaying a lung object associatedwith the inspired gas object, the lung object configured to represent apatients' lung status as sensed by a second sensor configured to detecta lung status; displaying an expired gas object configured to display apatient's respiration volume as measured by a third sensor configured todetect expired gas, the expiration object being associated with the lungobject; displaying an airway object extending from the lung object andrepresenting a constriction state of a patient's airway as sensed by afourth sensor configured to detect airway constriction; displaying aventilation object coupled to the airway object, the ventilation objectrepresenting tidal volume of the patient's pulmonary system, wherein thetidal volume of the patient's pulmonary system is sensed using a fifthsensor configured to detect pulmonary system tidal volume, and whereinpulmonary objects are displayed in combination to exhibit interactionbetween objects to display pulmonary states of the patient; anddisplaying a normal reference frame surrounding each of the pulmonaryobjects, wherein the normal reference frame represents expected normalpulmonary values.
 13. A medical diagnostic display method as in claim12, wherein the inspired gas object, lung object, expired gas object,airway object, and ventilation object are organized in a order thatrepresents the pulmonary system as compared to timing of the pulmonaryprocesses.
 14. A medical diagnostic display method as in claim 12,wherein each pulmonary object can expand past the normal reference frameor contract within the normal reference frame to show an abnormalpulmonary state.
 15. A method for displaying diagnostic patientpulmonary data in a three-dimensional display on a display screen,comprising the steps of: measuring a patient's inspired gas volume usinga first sensor configured to detect inspired gas volume and representingthe inspired gas volume as an inspired gas object; displaying a lungobject associated with the inspired gas object, the lung object beingconfigured to represent a patients' lung condition as sensed by a secondsensor configured to detect a lung condition; displaying an expired gasobject configured to display the patient's respiration volume asmeasured by a third sensor configured to detect respiration, wherein theexpired gas object is associated with the lung object; representing aconstriction state of an airway using an airway object extending fromthe lung object, wherein the constriction state of the airway is sensedusing a fourth sensor configured to detect airway constriction;representing tidal volume of a patient's pulmonary system using aventilation object coupled to the airway object, wherein the tidalvolume of the patient's pulmonary system is sensed using a fifth sensorconfigured to detect pulmonary system tidal volume, and whereinpulmonary objects are displayed in combination to exhibit interactionbetween objects to display pulmonary states of the patient; andproviding a normalization frame surrounding each of the pulmonaryobjects, wherein the normal reference frame represents expected normalpulmonary values.
 16. A method as in claim 15, wherein each pulmonaryobject can expand past the reference frame or contract within thereference frame to show an abnormal pulmonary state.